I ve slowly come to the conclusion that I should likely not work as much as I do. Looking back over my GitHub graphs, and feeling my own internal pressure to do more work is getting to the point of insanity. I can t remember the last time I took a whole day off. As a result, I m going to make a new rule for myself, and put it in a semi-public place to help with peer-pressure enforcing this. I m not going to use my computer (except in vary rare and urgent circumstances) on Sundays, from here on out. The exact rules are still up in the air (Is the Phone OK? Tablet?) Let s see how this goes! I ll start next week (or just stop doing things today)

I ve spent a bit of time this saturday Dockerizing Debile s slave / workers (for those who don t know about Debile yet, there are a few posts where I explain it) Many thanks to Tianon Gravi for his work in helping me out :) I created two images to test - the base image for any slave (paultag/debile-slave-base, and a preconfigured image for our test master server. This includes a script to preconfigure the slave worker (OpenPGP keys, username, password) that are passed to it via a tarball generated by debile-remote. Hopefully this makes deploying debile slaves easier :) I was able to do a test build of about 300 packages with the dockerized slave, and I think it s a huge step forward. Onward!

I ve been hacking (on and off) on a small bit of code (written in Hy) called Acid. So, all of this is *really* fluid, and I m going to change it s API and code (it s currently just one massive hack), but I think the idea has kinda jelled enough for me to chat a bit about. Acid is a DSL for writing event-driven Hy. Currently it s just using global events (that ll change soon), and it s a monumental hack (not sure how much I can do there), but it works pretty well. It s built on top of tulip / asyncio, which is an amazing new Python 3.4 standard library package for working with async code. Although asyncio is designed around network-based usecases, I ve been trying to shoehorn in an event system on top of it. It s going well so far. A basic Acid script looks something like:

(trip
  (on :startup
    (every 1 second (emit :clock-pulse nil)))
  (on :clock-pulse (print "."))
  (emit :startup nil))

which will print a dot ( . ) every second forever. Something a bit more advanced (fetch the MBTA Red line T information once a minute forever)

(defn get-endpoint-url [line]
  (.format "http://developer.mbta.com/lib/rthr/ 0 .json"
    (get  :red-line  "red"
          :blue-line "blue"  line)))
(trip
  ;; OK. Let's do some work with MBTA feeds.
  (on :update-feed
      ;; let's just update feeds on a cron.
      (print (.json (.get requests (get-endpoint-url event))))
      (emit :feed-updated event))
  (every 1 minute
         (emit :update-feed :red-line)))

Clearly, both of these examples are just as clear as their straight functional counterparts, but I m keen to see what I can do with temporal recursion for long-running daemons. I think my first job will be porting snitch's codebase to Acid. Let me know if you have any ideas!

Hello, World! I ve got two upcoming talks - one at LibrePlanet 2014 about OpenPGP, and one at PyCon US 2014 (in Canada) on Hy. Slides will be posted as I give the talks. Hope to see folks there!

After years under Mako s loving care, iron-blogger is now under new leadership. The triumvirate consists of Me, Molly and John. If any interesting Boston-area people want to join, just email us!

I took Valve s offer up (by way of the kickass Collabora folks) for some games. I can confirm that I got them, and it looks like it s working great. Just want to give a shout-out and express my gratitude. I know it s not something they had to do, so it s super cool that they wanted to. Thalve!

Hello, World! Docker s in Debian! Isn t that great! Let s all get happy! It s called docker.io, so go ahead and sudo apt-get install docker.io whenever you want :) The first two uploads have a few errors, which are 100% my fault. The first were a set of FTBFS bugs, which were a stupid error on my part. Thanks to olasd for catching the remaining bug, related to stripping the binaries. I m so sorry this happened, it slipped through as a result of me having a local docker binary in /usr/local, which I tested completely before uploading a totally different binary. I won t let it happen again. It should be fixed now. However, this comes with a warning. It appears as though systemd (which, for the record I adore) is allowing lxc-start unmount /dev/pts and friends, which causes a bunch of damage to the host. I ve filed the bug as bug #734813. Currently the outdated mount(1) is holding back a workaround. Hopefully we get util-linux updated so we can hack around this in the short-term. Hopefully the bigger issues get solved. So, if you re a systemd user, please hold off on using docker.io until we resolve this issue in Debian.

Good morning all my hungover friends. New Hy release - sounds like the perfect thing to do while you re waiting for your headaches to go away. Here s a short-list of the changes (from NEWS) - enjoy!

Changes from Hy 0.9.11
   tl;dr:
    0.9.12 comes with some massive changes,
    We finally took the time to implement gensym, as well as a few
    other bits that help macro writing. Check the changelog for
    what exactly was added. 
    
    The biggest feature, Reader Macros, landed later
    in the cycle, but were big enough to warrent a release on it's
    own. A huge thanks goes to Foxboron for implementing them
    and a massive hug goes out to olasd for providing ongoing
    reviews during the development.
    
    Welcome to the new Hy contributors, Henrique Carvalho Alves,
    Kevin Zita and Kenan B l kba . Thanks for your work so far,
    folks!
    
    Hope y'all enjoy the finest that 2013 has to offer,
      - Hy Society
    * Special thanks goes to Willyfrog, Foxboron and theanalyst for writing
      0.9.12's NEWS. Thanks, y'all! (PT)
    [ Language Changes ]
    * Translate foo? -> is_foo, for better Python interop. (PT)
    * Reader Macros!
    * Operators + and * now can work without arguments
    * Define kwapply as a macro
    * Added apply as a function
    * Instant symbol generation with gensym
    * Allow macros to return None
    * Add a method for casting into byte string or unicode depending on python version
    * flatten function added to language
    * Add a method for casting into byte string or unicode depending on python version
    * Added type coercing to the right integer for the platform
    [ Misc. Fixes ]
    * Added information about core team members
    * Documentation fixed and extended
    * Add astor to install_requires to fix hy --spy failing on hy 0.9.11.
    * Convert stdout and stderr to UTF-8 properly in the run_cmd helper.
    * Update requirements.txt and setup.py to use rply upstream.
    * tryhy link added in documentation and README
    * Command line options documented
    * Adding support for coverage tests at coveralls.io
    * Added info about tox, so people can use it prior to a PR
    * Added the start of hacking rules
    * Halting Problem removed from example as it was nonfree
    * Fixed PyPI is now behind a CDN. The --use-mirrors option is deprecated.
    * Badges for pypi version and downloads.
    [ Syntax Fixes ]
    * get allows multiple arguments
    [ Bug Fixes ]
    *  OSX: Fixes for readline Repl problem which caused HyREPL not allowing 'b'
    * Fix REPL completions on OSX
    *  Make HyObject.replace more resilient to prevent compiler breakage.
    [ Contrib changes ]
    * Anaphoric macros added to contrib
    * Modified eg/twisted to follow the newer hy syntax
    * Added (experimental) profile module

sorry for not posting, life s been getting in the way. Here s an update of what I did the last few weeks: I hacked more on Hy. A lot more. Tons of new stuff is coming through, and I got a slot to talk about Hy at PyCon US! I worked a bit on Debian stuff. I got Debile a bit more in shape, and did more NEW queue work. Hopefully I can get some more in. I just bought another keyboard to supplement my Das Keyboard - the Tex Beetle. I m thinking of getting some pimped out keycaps. Get at me if you know anything about this. Merry christmas!

Snitch is the first DSL I ve written in Hy. It may be the first Hy DSL ever, but I have nothing to prove that. Since I ve had a bit of time while the hard-working Alioth maintainers hack on getting vasks alive, I wrote Snitch (and set up the Debian view to be shown on fraked.debian.net - more to come!) Snitch's DSL looks like:

(rules "personal"
  (rule "pault.ag" pingable httpable)
  (rule "lucifer.pault.ag" pingable httpable))

Which expands out to quite a few lines that query the server s status using Snitch s informants, and insert the result into MongoDB. More to come!

I'm very happy: I was finally able to present a talk at a Free Software conference in Paran , Argentina Regina's hometown. Not only in Paran , but at the Vieja Usina culture center, half a block away from her parents' house. So, I must doubly thank Laura: First, for letting us know there would be a Free Software conference there, and second, for taking some pictures :- What was this conference? Conferencia Regional de Software Libre, organized by Grupo de Usuarios de GNU/Linux de Entre R os (GUGLER). Of course, flying to Argentina (and more specifically, to Paran , which is ~500Km away from the international airport) just for a one day conference was out of the question So I gave the talk by videoconference. Of course, given we will be travelling for the December vacations to Argentina, I expect to meet in person the GUGLER guys soon. I gave a single talk, mixing together two different topics: (my very personal take on) the Free Software philosophy and Debian's place in the Free Software universe. I had a very good time giving the talk, and while I was unable to look at my audience, I got reports saying they were happy and interested. I even got some mails from them, which makes me quite happy ;-) Now, one of the recurring points whenever I talk about Debian: I often tell people that I cannot tell them why they should use Debian instead of other distributions. My years testing every distribution I come across are long gone, and I nowadays am familiar with Debian only. But I also tell them that personally I gain nothing by having more Debian users in the world What I want to achieve is the next logical step: To have more people contributing to Debian. So, here is a great opportunity for interested people, specifically a group that often has a hard time finding a way to collaborate with Free Software projects. Today, Paul Tagliamonte published a call for proposals for Debian 8 (Jessie)'s artwork. So, given many people always want to find a way to contribute to Free Software without being a coder, here's a golden opportunity. You can look at the themes sent for Debian 7 as a reference; look also at the technical requirements for your artwork, and... Well, you have until early February to work on it!

(This is a reproduction of the mail sent to -announce for the planet readers) Aloha, hackers, As some of you notice from the slamn default background in wheezy, (BTW;
a round of applause for Adrien Aubourg for creating Joy!) we ask
for contributions to the debian-desktop team in the form of themeing. This is the official call for proposals for the Jessie cycle. Please
look on the wiki (thank you, Valessio Soares de Brito for creating
the page!) So, if you d like (or know someone who d like) to create a desktop look
and feel that will be seen by trillions of people, be sure to send in
your artwork ASAP! Breaking with tradition, I d like to have the re-themed desktop in place
well before freeze, to ensure we have time to properly integrate it. I m picking the arbitrary deadline of Feb 5th (subject to change) for
artwork to be submitted. I know this isn t a lot of time, but we can
iterate as we integrate. The artwork is usually picked based on which theme looks the most:
- Debian (I m not defining this since I think everyone has their own take
of what Debian means to them) - plausible to integrate without patching core software (As much as I love some of the insanely hot looking themes, some
would require heavy GTK+ theming and patching GDM/GNOME.) - clean / well designed (without becoming something that gets annoying to look at a
year down the road - I think Joy does a good job of this)
This year, do keep in mind the Debian logo *is* freely licensed, so feel
free to use it in your theme.

That thread


My feels:



So, I submitted the issue to tech-ctte.


Let s make this painless and let ctte deal with it :)

Now that GSOC is over and DebianDroid is in a more functional state i ve decided to finally publish the app to the Google Play Store and to F-Droid. That way I can get way more feedback about it plus it might already be useful for some people.

---

You can find a features and todo list in the readme in my github repo or alioth mirror. Special thanks to Hans-Christoph Steiner (my gsoc mentor) and to Paul Tagliamonte for their help and feedback along the way. If you want to help send me your feedback about the app via mail to onexemailx@gmail.com or via github or play store. I m looking for bugs, new feature ideas, suggestions for rest/soap/any other apis I can use to implement features in the todo list and also possibly for other icon or design ideas. That s all for now. :) Edit: I did a last minute rename of the app package to net.debian.debiandroid (since debian.com doesn t exist) and it was a bit more complicated that I expected so I had to republish the app in the play store. Just uninstall the old one and install it again from the link above. I changed the description of the old app as well so it should be clear which is which. Sorry for the inconvenience. The good thing is that I fixed some bugs/crashes as well.

On a regular basis, I find myself the odd-man-out when it comes to talking about how to work with Python on Debian systems. I'm going to write this and post it so that I might be able to point people at my thoughts without having to write the same email in response to each thread that pops up. Turns out I don't fit in with the Debian hardliners (which is to say, the mindset that pip sucks and shouldn't exist), nor do I fit in with the Python hardliners (which is to say apt and dpkg are out of date, and neither have a place on a Development machine). I think our discourse on this topic has become petty and stupid in general. Let's all try to step back and drop a bit of the attitude. pip doesn't suck, and neither does apt. The truth is, both sides are wrong. As with any subject, the real answer here is much more nuanced than either side presents it. I'm going to try and present my opinion on this, in the way that both my Pythonista self and my Debianite self see the issue. Hopefully I can keep this short, to the point, and caked with logic. The case for dpkg (the Debianite in me) In defense of dpkg and apt, imagine having to install python-gnome2 on all your systems when you install. It'd be hell on earth. Imagine having a user try to do this. It's insane to assume that end-users will be using pip for this purpose. pip is fun and all, but it's also installing 100% untrusted code to your system (perhaps as root, if you're using pip with sudo for some reason), and hasn't been reviewed for software freeness, which is something Debian (and Debian users) take seriously. This isn't even to mention the hell that pip wreaks on dpkg controlled files / packages. Try to remember how much of your system running (yes, right now) is running because of Python or Python modules. Try to imagine how much of a pain in the ass it'd be if you couldn't boot into GNOME to use nm-applet to connect to wifi to pip install something. I'm sure even the most extreme pip'er understands the need for Operating System level package management. Debian also has a bigger problem scope - we're not maintaining a library in Debian for kicks, we're maintaining it so that end user applications may use the library. When we update something like Django, we have to make sure that we don't break anything using it (although, to be honest, the fact that we package webapps is an entire rant for later) before we get to update it to the newest release. Hell, with a few coffees, I could automate the process of releasing a .deb with a new upstream release, 100% unattended. I won't, however, since this is an insane idea. Let's go over a brief list of things I do before uploading a new package:

1. Review the entire codebase for simple mistakes.
2. Review the entire codebase for license issues.
3. Review the entire codebase for files without source, and track down (and include source for) any sourceless files (such as pickle files, etc).
4. Get to know the upstream, get to know open bugs, write something using the lib, in case I need to debug later.
5. Install the package.
6. Test the package.
7. Work out any Debian package issues (this is easy).

Now, a brief list of things I do before I update a package:

1. Review the changes between the last uploaded version (in diff format, if it's sane, otherwise get the VCS and review), ensure all the above are still OK.
2. Review for Debian-local issues (such as how it will upgrade, using piuparts, and adequate, etc).
3. Check to make sure it won't break any reverse dependencies.
4. Review for bugfixes that I might need to bring back to the stable release.
5. Figure out if I should (or even can) backport the package, if API is stable.
6. Review for bugs (upstream or in Debian) that I need to mention in the debian/changelog.

Clearly, this isn't a quick-and-dirty task. It's not a matter of getting a package updated (technically), it's a much more detailed process than that. This is also why Debian is so highly regarded for its technical virtuosity, and why the ISS decided to deploy Debian in space, despite other commercial distros such as Red Hat, or Ubuntu, and community distros, such as Fedora or Arch. It's also not Debian's job to package the world in the archive. This is an insane task, and it's not Debian's place to do it. We introduce libraries as things need them, not because we wrote some new library that someone may find slightly useful at some point in the future. maybe. Upstream developers and language communities (not only Python here) tend to lose sight of why we're doing this in the first place, which is our users. This isn't some sort of technical pissing contest to see who can distribute the software in the best way. Debian-folk always keep end users as our highest priority. I quote the Debian Social Contract, when I say that Our priorities are our users and free software. No one's trying to get developers to use dpkg to create software. In fact, as you'll see below, I actively discourage using system modules for development. The case for pip (the Pythonista in me) In defense of pip, the idea that Debian will keep the latest versions of packages is insane. The idea that we can keep pace with upstream releases is nuts, and the idea that every upstream release on pypi is ready to ship is bananas. b-a-n-a-n-a-annas. As a developer, I don't want to support every release, and I surely don't want other people depending on some random snapshot. Often times, I'll put stuff up on pypi as a preview, or to release often, and solicit feedback without having to give out instructions on using a git checkout (it's also easier to have them try a version from pypi so I can cross-ref the git tag to reproduce issues when they file them) pypi is easy, ubiquitous and works regardless of the platform, which means less of my development time is spent packaging stuff up for platforms I don't really care about (Arch, Fedora, OSX, Windows), even though I value feedback from users on those systems. The effort it takes to release something is limited to python setup.py sdist upload, and it's in a place (and in a shape) that anyone can use it without having 10 sets of platform-local instructions. Even ignoring all the above, when I'm writing a new app or bit of code, I want to be sure I'm targeting the latest version of the code I depend on, so that future changes to API won't hit me as hard. By following along with my dependencies' development, I can be sure that my code breaks early, and breaks in development, not production. Upstreams also tend to not like bug reports against old branches, so ensuring I have the latest code from pypi means I can properly file bugs. Lastly, I prefer virtualenv based setups for development, since I'm usually working on many things at once. This often means version mismatches in libraries, which brings in API changes (another whole rant here as well). I don't want to keep installing and uninstalling packages to switch between the two projects, and using a chroot(8) means a lot of overhead and that it's disconnected from my development environment / filesystem, so I resort to virtualenv to isolate my Development environment. Final notes I don't want to keep arguing about this. Just accept that the world's a big place and that there exist use-cases that both apt and pip need to exist and work in the way they're working now. At the very least, try and understand there exist smart people on both sides, and no one is trying to screw anyone over or keep their own little private club to themselves. Hopefully, going forward, we can make sure that the integration between these two tools gets better, not worse. Help make this dream a reality. Contribute to a productive tone, not a destructive one. In short:

• Use pip without sudo always. Don't tell people to use sudo.
• Use apt or dpkg when deploying system-wide.
• Understand people are going to package, and they will be more concerned about software using your library then keeping your library up to date.
• Understand Debian Developers and package maintainers have to do a lot of work when updating or sponsoring an upload.
• Understand upstream developers can't be bothered to fix every issue with every release (release early, release often) with some snapshot you introduced into unstable.
• Use pip and virtualenv in development setups, so we can upgrade your app when we upgrade the lib.

I ll be at the Google Summer of Code Mentor s Summit this year - looking forward to meeting with any other Debian-folk (or non-Debianfolk!) who happen to find themselves at the summit. See y all soon!

It s been a long time since I last talked about libolla, but it s one of those things I use everywhere and never bothered to
package up and upload. Well. I packaged it. And uploaded it. Finally. Now, let s look at the new quickstart guide: (this assumes you use bash, since I decided to abuse bash_completions to insert an LD_PRELOAD. I know this is gross, but I totally need a better idea) Now, look on in dumbfounded obviousness as I show you what doesn t work:
$ ssh master.d.o
ssh: Could not resolve hostname master.d.o: Name or service not known
Sadsies. Now, let s make it work:
$ ssh master.d.o whoami
ssh: Could not resolve hostname master.d.o: Name or service not known
$ sudo apt-get install -t experimental olla
# (restart your terminal)
$ ssh master.d.o whoami
paultag
Thank you, thank you! I ll be here all week! Anyway. Have fun, kids! Remember, hacks are fun!

I ve been pretty absent from the net over the last 3 weeks, (and for the first time in well over a year, I missed a week of blogging) I ve been pretty busy with real-life-stuff, nothing ultra interesting to report, I ve been working away with the ftpteam more (as I m sure some of you have noticed), and doing Hy work when I can. Nothing major on either front yet :)

Long time since I published the last update about Tanglu. So here is a short summary about what we did meanwhile (even if you don t hear that much, there is lots of stuff going on!) Better QA in the archive We now use a modified version of Debian s Britney tool to migrate packages from the newly-created staging suite to our current development branch aequorea . This ensures that all packages are completely built on all architectures and don t break other packages. New uploads and syncs/merges now happen through the staging area and can be tested there as well as being blocked on demand, so our current aequorea development branch stays installable and usable for development. People who want the *very* latest stuff, can embed the staging-sources into their sources.list (but we don t encourage you to do that). Improved syncs from Debian The Synchrotron toolset we use to sync packages with Debian recently gained the ability to sync packages using regular expressions. This makes it possible to sync many packages quickly which match a pattern. Tons of infrastructure bugfixes The infrastructure has been tweaked a lot to remove quirks, and it now works quite smoothly. Also, all Debian tools now work flawless in the Tanglu environment. A few issues are remaining, but nothing really important is affected anymore (and some problems are merely cosmetic). Long term we plan to replace the Jenkins build-infrastructure with the software which is running Paul Tagliamontes debuild.me (only the buildd service, the archive will still be managed by dak). This requires lots of work, but will result in software not only usable by Tanglu, but also by Debian itself and everyone who wants a build-service capable of building a Debian-distro. KDE 4.10 and GNOME 3.8 Tanglu now offers KDE 4.10 by default, a sync of GNOME 3.8 is currently in progress. The packages will be updated depending on our resources and otherwise just be synced from Debian unstable/experimental (and made working for Tanglu). systemd 204 & libudev1 Tanglu now offers systemd 204 as default init system, and we transitioned the whole distribution to the latest version of udev. This even highlighted a few issues which could be fixed before the latest systemd reached Debian experimental. The udev transition went nicely, and hopefully Debian will fix bug#717418 too, soon, so both distributions run with the same udev version (which obviously makes things easier for Tanglu ^^) Artwork We now have a plymouth-bootscreen and wallpapers and stuff is in progress Alpha-release & Live-CD ? This is what we are working on we have some issues in creating a working live-cd, since live-build does have some issues with our stack. We are currently resolving issues, but because of the lack of manpower, this is progressing slowly (all contributors also work on other FLOSS projects and of course also have their work ) As soon as we have working live-media, we can do a first alpha release and offer installation media. You! Tanglu is a large task. And we need every help we can get, right now especially technical help from people who can build packages (Debian Developers/Maintainers, Ubuntu developers, etc.) We especially need someone to take care of the live-cd. But also the website needs some help, and we need more artwork or improved artwork In general, if you have an idea how to make Tanglu better (of course, something which matches it s goals ) and want to realize it, just come back to us! You can reach us on the mailiglists (tanglu-project for stuff related to the project, tanglu-devel for development of the distro) or in #tanglu / #tanglu-devel (on Freenode).

The LLVM toolchain version 3.3 has been released a couple months ago.
Here are now the result of the rebuild of Debian archive using this version of the compiler.
Like the previous releases, we are at a bit less than 12% of packages failing (2188 packages on a total of 18854).
More warnings / errors detections have been added to the software (for example: like this defect of the C++ standard or the detection of unused linker option) causing more build failures, but, in the mean time, we fixed some issues in the Debian packages... As usual, the following image shows clearly the evolution of the build failures over time. As stated in my blog post for the release 3.2, this rebuilds prove that Clang is ready for production in term of support of the C, C++ and Objective C languages. With the performance results showed by Chandler Carruth from Google at the last Euro LLVM summit (see this video from 5:40), I believe that it is now time to report and fix the bugs in the upstream packages. I also presented this work (video) at the Debconf 13 last week in Vaumarcus (Switzerland) and I will be also presenting this work at the Linux Plumbers Conference, New Orleans. With L o Cavaille (as part of his GSOC) and Paul Tagliamonte, we are also working on providing a better automatic rebuild infrastructure for clang-built packages (and other static analyzers). More in the next few weeks. Finally, I would like to thank folks at AWS for the Debian credit and David Suarez for helping on with the Ruby segfaults.